Multi-frame Collaboration for Effective Endoscopic Video Polyp Detection via Spatial-Temporal Feature Transformation

Precise localization of polyp is crucial for early cancer screening in gastrointestinal endoscopy. Videos given by endoscopy bring both richer contextual information as well more challenges than still images. The camera-moving situation, instead the common camera-fixed-object-moving one, leads to significant background variation between frames. Severe internal artifacts (e.g. water flow human body, specular reflection tissues) can make quality adjacent frames vary considerately. These factors hinder a video-based model effectively aggregate features from neighborhood and give better predictions. In this paper, we present Spatial-Temporal Feature Transformation (STFT), multi-frame collaborative framework address these issues. Spatially, STFT mitigates inter-frame variations situation with feature alignment proposal-guided deformable convolutions. Temporally, proposes channel-aware attention module simultaneously estimate correlation adaptive aggregation. Empirical studies superior results demonstrate effectiveness stability our method. For example, improves image baseline FCOS \(10.6\%\) \(20.6\%\) on comprehensive F1-score task CVC-Clinic ASUMayo datasets, respectively, outperforms state-of-the-art method \(3.6\%\) \(8.0\%\), respectively. Code available at https://github.com/lingyunwu14/STFT.